Papers with inter-task collaboration
Enhancing Multimodal Continual Instruction Tuning with BranchLoRA (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to fine tune Multimodal Large Language Models (MLLMs) are prone to Catastrophic Forgetting (CF) existing approaches rely on the Mixture-of-Experts (MoE) LoRA framework to preserve previous instruction alignments. |
| Approach: | They propose an asymmetric tuning-freezing mechanism to mitigate parameter inefficiency . branch-specific routers are introduced to ensure optimal branch distribution over time . |
| Outcome: | The proposed framework outperforms existing frameworks on the latest MCIT benchmarks. |
Task-Specific Information Decomposition for End-to-End Dense Video Captioning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for label assignment fail to ensure relevance of localization information to descriptions. |
| Approach: | They propose a decomposed dense video captioning framework that derives localization and captioning queries from event queries, enabling task-specific representations while maintaining inter-task collaboration. |
| Outcome: | Experiments on YouCook2 and ActivityNet Captions show that the proposed framework achieves state-of-the-art performance. |